Description

This is the parser admission and fairness part of the Pipe resilience work.

Global and per-Pipe-region limits

• Add pipe_tsfile_parser_in_flight_max_num, defaulting to half of the available processors with a minimum of 1.
• Add pipe_tsfile_parser_in_flight_max_num_per_pipe_region, defaulting to 1.
• Enforce both limits even when Pipe memory accounting is disabled. Since every admitted parser reserves pipe_tsfile_parser_memory, the per-Pipe-region limit also provides an approximate parser memory quota.
• Scope the local limit by (pipeName, creationTime, dataRegionId), so different regions of one Pipe can parse concurrently while TsFiles from the same Pipe region remain bounded.
• Document both properties in iotdb-system.properties.template and support hot reload. Increasing either limit wakes the next eligible queued reservation immediately.

Hierarchical fair admission

PipeMemoryManager schedules waiting requests in three levels:

1. Round-robin across Pipes.
2. Round-robin across waiting regions within the selected Pipe.
3. FIFO across TsFiles within the selected Pipe region.

After a Pipe obtains one parser, its remaining requests move behind other waiting Pipes. After a region obtains one parser, its remaining requests move behind other waiting regions of the same Pipe. A multi-region Pipe therefore keeps region parallelism without receiving extra weight in the cross-Pipe fairness queue.

Each TsFile event owns a stable reservation key and carries its stable DataRegion identity through reservation, cancellation, release, and resource finalization. Timeout, explicit close, and resource finalization cancel pending reservations so an abandoned request cannot block a queue head.

Waiting events block on their reservation keys instead of polling every 10 ms. Admission, parser release, request cancellation, memory release, and parser-limit hot reload signal only the next eligible reservation. The signal is retained until consumed, avoiding a lost wake-up between the failed reservation attempt and the wait.

When only hard-threshold memory headroom remains, the selector skips Pipes that already own a parser and admits a waiting Pipe with no parser first.

Diagnostics

An unmatched TsFile parser memory release is treated as an invariant violation and emits a WARN containing the Pipe identity and DataRegion.

Tests

• Verify global and per-Pipe-region limits together.
• Verify different regions of one Pipe can hold parsers concurrently while the same region remains limited.
• Verify round-robin admission across Pipes and across regions within one Pipe.
• Verify a Pipe with many waiting regions does not receive more cross-Pipe scheduling weight.
• Verify hard-threshold headroom goes to a Pipe that has no active parser.
• Start nine concurrent TsFile reservations from three Pipes against one parser slot; wait on directed notifications (without polling), verify every TsFile is eventually admitted, and verify the first three admissions come from different Pipes.
• Verify both parser limits take effect through DataNode hot reload.

This PR has:

• been self-reviewed.
  • concurrent read and write
• added comments explaining non-obvious scheduling decisions.
• added unit tests for the new admission paths.

Key changed/added classes

• PipeMemoryManager
• PipeTsFileInsertionEvent
• PipeMemoryManagerTest
• PropertiesTest